Papers with manual prompt engineering
Reflection in the Dark: Exposing and Escaping the Black Box in Reflective Prompt Optimization (2026.acl-srw)
Copied to clipboard
| Challenge: | Automatic prompt optimization (APO) is a powerful paradigm for improving LLM performance without manual prompt engineering. |
| Approach: | They propose a framework that decouples hypothesis generation from prompt rewriting . they propose VISTA framework that recovers accuracy to 87.57% on same defective seed . |
| Outcome: | The proposed framework outperforms baselines on GSM8K and AIME2025 on a defective seed. |
To Clarify or not to Clarify: A Comparative Analysis of Clarification Classification with Fine-Tuning, Prompt Tuning, and Prompt Engineering (2024.naacl-srw)
Copied to clipboard
| Challenge: | Xu et al., 2019) show that pre-trained language model fine-tuning and prompt tuning are better than manual prompt engineering for clarification identification. |
| Approach: | They propose to use pre-trained language model fine-tuning, prompt tuning and manual prompt engineering to model clarification identification. |
| Outcome: | The proposed model outperforms pre-trained language model fine-tuning, prompt tuning and manual prompt engineering on the task of clarification identification. |
Beyond One-Size-Fits-All: Inversion Learning for Highly Effective NLG Evaluation Prompts (2026.tacl-1)
Copied to clipboard
| Challenge: | Evaluating natural language generation systems is challenging due to the diversity of valid outputs. |
| Approach: | They propose an inversion learning method that learns effective reverse mappings from model outputs back to their input instructions. |
| Outcome: | The proposed method requires only a single evaluation sample and eliminates manual prompt engineering. |
LingVarBench: Benchmarking LLMs on Entity Recognitions and Linguistic Verbalization Patterns in Phone-Call Transcripts (2026.eacl-industry)
Copied to clipboard
| Challenge: | Existing methods degrade under disfluencies, interruptions, and speaker overlap, yet large real-call corpora are rarely shareable. |
| Approach: | They propose a benchmark and semantic synthetic data generation pipeline that generates linguistically varied training data via (1) LLM-sampled entity values, (2) curated linguistic verbalization patterns covering diverse disfluencies and entity-specific readout styles, and (3) a value–transcript consistency filter. |
| Outcome: | The proposed pipeline outperforms zero-shot baselines and matches or closely approaches human-tuned prompts on real customer transcripts. |
I-SEE: An Instruction-tuned, SOP-Enhanced Quality Evaluator for Product Content (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing approaches to content evaluation treat information uniformly without prioritizing based on customer relevance. |
| Approach: | They propose a framework that combines domain expertise with a single instruction to improve content. |
| Outcome: | a new framework outperforms existing models in detecting inconsistencies across 20 product categories and 150 product specific features. |
Model Performance-Guided Evaluation Data Selection for Effective Prompt Optimization (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing prompt engineering methods rely on randomly selected evaluation subsets, leading to suboptimal prompts. |
| Approach: | They propose an iterative evaluation data selection approach for effective prompt optimization using real time model performance. |
| Outcome: | The proposed approach improves effectiveness by 1.6% to 3.1% and stability by 50% to 55.5% on two datasets BIG-bench and LIAR and two models GPT-3.5 and GPT-4o-mini. |
MulVul: Retrieval-augmented Multi-Agent Code Vulnerability Detection via Cross-Model Prompt Evolution (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) struggle to automate real-world vulnerability detection due to the heterogeneity of vulnerability patterns and manual prompt engineering for massive weakness categories is unscalable. |
| Approach: | They propose a retrieval-augmented multi-agent framework for precise and broad-coverage vulnerability detection using a coarse-to-fine strategy. |
| Outcome: | The proposed framework outperforms the baseline model on 130 CWE types and achieves 34.79% Macro-F1 performance. |
Uncovering Sentiment Analysis Circuit in Large Language Model (2026.acl-long)
Copied to clipboard
| Challenge: | Prior work has shown that sentiment is encoded linearly in LLM representations, but their ability to utilize this information remains fragile to prompt variations. |
| Approach: | They propose a simple inference-time intervention method that amplifies circuit features to compensate for insufficient activation. |
| Outcome: | The proposed method improves on a sentiment analysis circuit with sparse autoencoders and circuit-level analysis. |
PromptWizard: Optimizing Prompts via Task-Aware, Feedback-Driven Self-Evolution (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have transformed AI across diverse domains, with prompting being central to their success in guiding model outputs. |
| Approach: | They propose a framework for discrete prompt optimization that generates human-readable prompts using feedback-driven critique and synthesis process. |
| Outcome: | The proposed framework improves prompt quality across 45 tasks and reduces API calls, token usage and overall cost. |
IntCoOp: Interpretability-Aware Vision-Language Prompt Tuning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing prompt-tuning frameworks lack interpretability, limiting their ability to understand compositional nature of images. |
| Approach: | They propose a prompt-tuning method that integrates compositional attributes into manual prompts to enhance image-text alignment scores. |
| Outcome: | The proposed method improves CoOp performance by 7.35% across 10 diverse datasets. |